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Abstract 

A partial differential equation is derived, describing the replicator dy- 
namics with mutations of games with a continuous strategy space. This 
equation is then applied to continuous versions of symmetric 2x2 games, 
such as the Prisoners Dilemma, Hawk-Dove and Coordination games, and 
to the Ultimatum Game. In the latter case, we find that adding even a 
small mutation term to the replicator equation leads to a solution where 
the average offer is significantly larger than zero. 

1 Introduction 

Evolutionary game dynamics is a fast developing field, with applications in biol- 
ogy, economics, sociology and anthropology. Background material and countless 
references can be found in the monographs by Weibull 1 , Fudenburg & Levine 
0, Samuelson [jj], Hofbauer & Sigmund 0], Gintis [5], Cressman jH] and Vincent 
& Brown [7] or in the survey paper by Hofbauer & Sigmund [Sj. The standard 
ingredients of evolutionary game dynamics are a population of players, an n- 
person game, a set of strategies and a rule to update the distribution of strategies 
from one generation to the next. Within this general setting several variations 
are possible: time can be discrete or continuous, populations can be finite or 
infinite, the game can have a finite or infinite number of strategies. Also, there 
are several choices for the updating rule, the most popular of which are Adap- 
tive Dynamics (See e.g. Diekmann and Replicator Dynamics, introduced 
by Taylor & Jonker |l(Jj . In this paper we consider a model with continuous 
time, an infinite population and a 2-person game, where each participant has a 
continuum of strategies to choose from. The update rule we use is the replicator 
dynamics, with deterministic mutations. Although this model has been alluded 
to by Dieckmann where a hierarchy of evolutionary models is presented, 
its details have not been worked out before. 
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The strategy space that we consider is a subset of R n , which in all appli- 
cations is compact. In section 2 we first consider replicator dynamics with a 
deterministic mutation term on a finite set of strategies. Using a procedure that 
is familiar in Statistical Mechanics, we make a transition to an infinite, contin- 
uous, strategy-space. The state-space of the model is then no longer discrete, 
but consists of distributions over the strategies. We derive a partial differential 
equation, together with appropriate boundary conditions, that describes the 
time-evolution of this distribution, given an initial distribution. Without the 
mutation term, the equation we derive was previously studied by Cressman 
and Oechsler & Riedel 0m addition of a mutation term is new and leads 
to completely different dynamics. 

In section 3, the equation is applied to symmetric 2x2 games, where we need 
to extend the original two-strategy set to a continuous one. This can be done 
by allowing for strategies that play one of the pure strategies with a certain 
probability. We compare our results with those of Vaughan who analyses 
the replicator dynamics for the pure strategies, to which a stochastic perturba- 
tion term is added, leading to a Fokker-Planck equation. There are similarities 
between the two models, but also striking differences. In the stochastic muta- 
tion version, there is always only one attracting, stationary, distribution, which 
for small mutations converges to a point-distribution (Dirac-delta function) . In 
other words, all players eventually use the same strategy. In the deterministic 
mutation version, however, we find the possibility of two attracting stationary 
distributions existing simultaneously. Also, in certain cases the limiting distri- 
bution for small mutations is not concentrated on a point, but has the whole 
strategy space as support. 

A more complicated example, namely the Ultimatum Game, is treated in 
section 4. In this game, a sum of money is to be split by two players. The first 
player proposes the split and the second player then has the choice to accept the 
split or refuse, in which case both players get nothing. The solution offered by 
standard game theory is for the second player to accept any amount (something- 
is always better than nothing) and for the first player, therefore, to offer the 
lowest possible amount. In our model we find that the offers converge to a 
Gauss-like distribution around a mean that is not equal to zero. The position of 
the mean and the width of the distribution seem to converge to zero when the 
rate of mutation vanishes. However, this convergenge is slow, so that even for 
very small values of the mutation rate, the average offer is well above zero. Also, 
the dynamics shows two time-scales. A random starting distribution is initially 
attracted to a member of a certain set of distributions, which can have an average 
offer much larger than zero. Then, on a time-scale inversely proportional to the 
rate of mutation, the solution converges to the, unique, stationary solution. 

In section 5 we discuss the results and suggest some topics for further re- 
search. 
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2 From the discrete to the continuous equation 



The equation for the replicator dynamics with continuous strategy space will be 
derived by a limiting process, starting from the equations for the discrete case. 

Following Hofbauer and Sigmund 0], we consider an infinite population of 
players and a set of strategies Si, - ■ ■ , Sn- When a player A opts for strategy 
Si against B, who uses Sj, the payoff to A is taken to be Mij, and the payoff to 
B is Mji . Consequently, if pj (t) is the fraction of the whole population that at 
time t plays the strategy Sj , the average payoff to each player using Si is equal 
to 

JV 

HWE^^ft. (1) 

3=1 

The average payoff for the whole population therefore is 

N N 

TT(p) = J2 n * (v)Pi = Pi M ijPj ■ ( 2 ) 



Now in the course of time, the fraction p. L changes at a rate which is proportional 
to the difference between the payoff to pi and the average payoff for the whole 
population. In addition we assume that for each player of Sj there is a transition 
probability per unit time to make a spontaneous transfer to strategy St at a 
rate given by Wij . In this way the discrete replicator equation with mutation is 
derived: 

N 

Pi = (Ui( P ) - u(p)) Pi + Y,( w m - w aPi)- (3) 

3 = 1 

N 

It is easy to show that S(t) = J2 Pitt) = 1 for a11 1 if S (°) = L 

i=i 

In transforming to a continous strategy space, we replace the discrete index 
i by a continuous variable s £ D C R n and the variables pi (t) by a probability 
distribution P(s,t). The payoff matrix My must now be replaced by a payoff 
function M(s, s'), which gives the payoff to strategy s when playing strategy s'. 

Eq.© now takes the form 

^A^(n( s ,p)-n(p))p( s ,t) + M(p,s) (4) 

in which the mutation term is equal to 



M(P,s) = J [W(s\s')P{s',t)-W(s'\s)P{s,t)]ds'. (5) 
The average payoff for strategy s and the total average payoff are given by 

n(s,P) = J M(s,s')P(s',t)ds' (6) 
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and 

H(P)= J U(s,P)P(s,t) ds. (7) 

Let us now restrict ourselves to mutations in which only small changes in the 
strategies occur and apply the method which is used to derive the Fokker-Planck 
equation from a master equation |12| . For simplicity of presentation, we restrict 
ourselves to the case of a one-dimensional strategy space. Define £ by s' = s — £ 
and write the mutation rate as a function of s and £ 

W(s'\s) = W(s,s-s') = W(s,£) (8) 

and so 

W(s\s') = Wis', s' -s) = W(s - £, -£)• (9) 

Assume now that W(s,£,) varies slowly in the first variable s and that due to 
the mutations only small variations in the strategies will occur. Then W(s,t;) 
is only nonvanishing when £ is small. In the mutation term (J5J, which can be 
written as 

M(P, s)=f [W(s - -t)P(s - £, t) - W(s, OP(s, t)} dC, (10) 

we can now expand the dependence on the first variable in powers of £ and 
obtain 

M(P,s) = [{W(s > -0P(s,t)-^{W(s,-0P(s,t)}+ (11) 

+ ^ 2 £ W S ' -Z)P(s, t)} + -~- W(s, QP{s, t)\ de 

Because f[W(s,—£) — W(s, £)]P(s, t)d£ = 0, the first and last term cancel, so 
that to second order we are left with 



in which 



•M( p > s ) = —AMs)P(s,t)} + ^^{a 2 (s)P(s,t)}, (12) 



a 1 (8) = -[tW(a,$dZ and a 2 {s) = J ?W(s, d£. (13) 

We will further simplify the equations by assuming that the average change in 
strategy due to mutations is equal to zero, so a>i(s) = 0, and that the average 
of the square of this change is constant, so cx2(s) — 2a. 

The final form of the continuous replicator equation Q then becomes 

= ( n( s , P (t)) n(p(t)))P(s, t) + *ap(s, t), (w) 

where we have restored the correct dimensionality of the strategy space by 
replacing -S-^ by the n-dimensional Laplace operator. P(s,t) should satisfy 
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P(s, t) > and S(t) = J D P(s, t) ds, should be equal to unity at all times. This 
last condition is fulfilled when we choose Neumann, or reflecting, boundary 
conditions: 

n.VP(s,t)\ eD =0, (15) 

where n is the normal to the boundary dD of the domain D. 
Indeed, integrating (|T4"|) over D and using l(TB|) we find 

^M = n(p)(i-S(t)), (i6) 

showing that if S(0) — 1 then S(t) = 1 for all times. In the case that a = 0, 
condition l|15l) is not required to ensure that S(t) remain constant in time. 

Equation (|14l) is a nonlinear reaction-diffusion equation, where the reac- 
tion term II(s, P(t)) — H(P(i)) is nonlocal. On the function space of twice 
continuous space-differentiable and once timc-diffcrcntiable functions, we can 
show that the solution of (|14(l exists for all times. This follows from the as- 
sumption that M(s,s') is bounded on D, so that \H(s, P(t)) ~ H(P(t))\ < 
J DxD \M(s, s')\P(s, t)P{s\ t) ds ds' + J D \M(s, s')\ P(s>, t) ds' < 

max \M(s, s')\ { f DxD P(s, t)P(s', t) ds ds 1 + J D P(s', t) ds'} 
= 2zaax\M(s,s')\. 

Standard comparison theorems for parabolic equations (Pao complete 
the proof. Also, by standard positivity results for parabolic equations, it can 
be shown that the when the initial distribution P(s, 0) > 0, then P(s, t) > for 
all times t. 

Numerical simulations suggest that even stronger results hold. In particu- 
lar, we suspect that the solution of Eas. l|T4^ . lfT31) is uniformly (in space and 
time) bounded in terms of the sup-norm of the initial distribution. For one- 
dimensional strategy spaces, we speculate that the solution will always converge 
to a stationary solution. 

For a — 0, Eq. (|14fl has been studied by Cressman [2! and Oechsler & Riedel 
|15| . They show that Ea. (|14f> has a unique solution, for all times, on a large 
space of distributions, containing amongst others the Dirac-delta distributions. 
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3 Symmetric 2x2 games 



In symmetric 2x2 games there are two possible strategies, denoted by I and 
//. The payoff to player A is given by the payoff matrix 



A\B 


I 


II 


I 


a 


b 


II 


c 


d 



3.1 Discrete replicator dynamics 

The discrete replicator dynamics associated with this game consists of an infinite 
population where a fraction x\ (t) plays the pure strategy I and a a fraction xi (t) 
plays the pure strategy 77 . The payoffs to strategies I and II are given by: 

11/ (xi, X2) — ax 1 + bx2 and II//(xi, x%) = cx\ + dx%- (17) 

The average payoff to the total population is then 

II(xi, x%) — Ui(xi,X2)xi + II//(xi,X2)x2 — ax\ + (b + c)x\X2 + dx\. (18) 

With a mutation rate matrix of the form VT^cr^'j' ^ , a > and using 
x\(t) + X2(t) = 1, this leads to the following equation for x\(t) : 

xi = xi (1 - xi)(B + {A- B)xx) + a(l - 2xi), (19) 

where A = a — c and B = b — d. The solutions of Eq. l|19|) for a = are 
summarised in figure 1. 
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Figure 1: The four quadrants of the parameter space 

A typical example of a game in the quadrant A > 0, B > is the classic 
Prisoners Dilemma, where a — 1, b = 5, c = 0, d = 3. The strategy I corresponds 
to 'defect' and strategy II to 'cooperate'. Figure 1 shows that for a = the 



G 



discrete replicator dynamics Eci. ((T§|l predicts a final outcome of x\ = 1, X2 = 0, 
or 'All Defect'. The effect of the mutation is to shift the stable solution to a 
slightly lower value of x±. 

The Hawk-Dove game (also known as Chicken) has a — (G — C)/2,b = 
G, c = 0, d = G/2, where now strategy I corresponds to 'hawk' (or 'never back 
down') and 77 to 'dove' ('allways back down'). When the cost C to the loser of 
a hawk-hawk fight is larger than the gain G a hawk makes when confronting a 
dove, we have A < 0, B > 0. It follows from Ea. (|19|l that for a = the solution 
tends to the stable equilibrium x\ = Bj (B — A) = G/C, describing a population 
where the strategies co-exist. Also in this case, the effect of the mutation term 
is restricted to a small shift in the location of the stable equilibrium. 

The quadrant with A > 0, B < is the domain of the Coordination Games, 
exemplified by the situation a = 2, b = 0, c = 0, d = 1. In a Coordination Game 
it is advantageous for both players to play the same strategy. In a 2 x 2 game, 
this leads to two equilibria. Note, however, that in the example mentioned here 
the situation where both play strategy / is superior to the one where both play 
77. For cr = there are two stable solutions and one unstable one, and the 
final outcome depends on the initial situation. When xi(0) < -| the solution 
will ultimately tend to x\ = 0, otherwise to x\ = 1. In other words, when the 
initial fraction of strategy / players is too small, the final population will consist 
exclusively of strategy 77 players, even though this is the less attractive of the 
two equilibria. 

In the case of Coordination Games, the effect of mutation can qualitatively 
change this picture. With the above given values of a, b, c and d, we have plotted 
the right-hand side of Ea. H19(l for several values of a. (figure 2). 




Figure 2: The bifurcation property of Eg. (|19|l 



From this picture it follows that for values of a larger than a critical value 
a c (in this example <j c « 0.11) the only equilibrium left is the optimal one near 
x\ = l, which is also the globally attracting solution. 
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3.2 Discrete replicator dynamics with a stochastic term 

In [T5|, Ea. (|19l) is studied, where instead of a deterministic mutation term, a 
small noise term is added. This leads to the stochastic equation: 

dx = G(x)dt + 2adW, (20) 

where G(x) = x(l — x)(B + (A — B)x) and W(t) denotes a Wiener process with 
zero mean and unit variance. 

In this model, the mutation from one strategy to another does not happen at 
a fixed rate, as in the model described by Ea . (|19|) . but rather the fraction X\(t) 
is changed by a small random amount per time-step. To describe the outcome 
of Ea. H2(Jf) . we consider the evolution of the probability density f(x,t). The 
probability that at time t the fraction of strategy / players lies in the interval 
[x, x + Ax) is given by f(x, t) Ax. The equation for f(x, t) is the Fokker-Planck 
equation |T2|: 

2^-A(oW^.» + ^. P.) 

We assume reflecting boundaries, which yield as boundary conditions: 
df(x = 0,t) _ df(x = l,t) 



dx dx 



= 0. (22) 



Eg. 1^2111 with conditions (|22|l has a unique, attracting, stationary distribution 

f*( x ) ng. 

It is easy to see that this equilibrium distribution is given by 

f*{x) = C*cxp(- / G(x')dx'), (23) 
& Jo 

where the constant C is determined by the condition f Q f*(x)dx = 1. 
Differentiating f*(x) once yields 

df*(x) C„,. A 



dx 



-G(x)exp(- I G(x')dx'), (24) 
a a Jo 



from which it follows that the extrema of f*(x) are the zeroes of G(x), which are 
the equilibria of the discrete replicator equation (|19|l with a = 0. Differentiating 
once more gives 

d 2 P(x) C,w, v , G 2 {x) 



dx 2 



C G 2 (t) 1 f 

-(G , (x) + —^-)exp(- G(x')dx'), (25) 
a a a J 



so at an equilibrium point x e we have that d r ^ x " ] = KG'{x e ) with K > 0. 
Therefore, a stable equilibrium of Ea. (|19|) with a = corresponds to a maximum 
of f*(x), and an unstable equilibrium to a minimum. 

In the limit a — ► 0, the stationary distribution f*(x) will tend to a point- 
distribution, where the total probability is concentrated on one point. In the 
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case of the Hawk-Dove Game and the Prisoners Dilemma it is clear that this 
point-distribution is concentrated on the unique stable equilibrium of the corre- 
sponding discrete replicator equation. For Coordination Games, there are two 
stable equilibria in the discrete case. Correspondingly, the stationary distri- 
bution has two local maximuma, namely at x = and at x = 1. In it 
is proved that for a — ► one of these two maxima will eventually dominate. 
The equilibrium that finally emerges is the one which has the largest basin of 
attraction in the discrete case, and is known in the game-theory literature as 
the risk-dominant equilibrium. 

3.3 Continuous replicator dynamics 

The two strategy set of 2 x 2 games can be extended to a continuum of strate- 
gies, each of which indicated by a real number xe [0, 1]. For the payoff function 
M(x, x') we choose a simple interpolation between the four payoff values of the 
symmetric 2x2 game: 



This game can be considered as the underlying discrete 2x2 game where now 
mixed strategies are allowed, in the following sense. A strategy xe[Q, 1] means 
that the player will use pure strategy / with probability x. We now assume 
that two players, using strategies x and x' respectively, at one encounter play 
each other a large number of times. Ea. (|26fl then gives the expectation value 
of the payoff to the first player. Because our players will soon become aware of 
the Law of Large Numbers, they won't bother with playing against each other, 
but at an encounter simply settle for the payoff given by Ea. i|26|) . making this 
a deterministic game. 

The expressions for II(x, P) and II(P) are easy to calculate: 



M(x, x') = xx'a + x(l - x')b + (1 - x)x'c + (1 - x)(l - x')d. 



(26) 




I {xx'a + x(l - x')b + (1 - x)x'c + (1 - x)(l - x')d)P(x', t) dx' 



Jo 



d + (b - d)x + (c - d)x{t) + (a - b - c + d)xx{t) 



in which 




(27) 



is the average strategy. Then 




d + (b + c - 2d)x(t) + {a - b - c + d)x 2 (t), 
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and 

Il(x, P) - II(P) = (b-d)x-(b-d)x(t) + {a-b-c + d)(xx(t)-x 2 (t)) 
= (B + (A-B)x(t))(x-x(t)), (28) 

where A and -B are the same parameters as defined in section 3.1. 
Ea. l|14|) for P(x,t) now becomes: 

= (B + (A - B)x(t)){x m)P(*, t) + ^ P qX^ (29) 
with boundary conditions: 



f dP{x.t) \ _ f dP{x,t) 



V <9x ; a=0 V ^ /x=i 



= 0, (30) 



and an intial distribution P(x, 0). 

3.3.1 The equation without mutation 

As was noted before, when considering Eq. l|29|l without mutation, i.e. with 
a = 0, it is not necessary to impose the boundary conditions l|30|) in order to 
ensure that J Q P(x, t) dx remain constant. In ^3] it is shown that 

^A = {B + {A-B)x{t)){x-x{t))P{x,t), P(x,0)=P (x) (31) 

has a unique solution for all t > 0. In this subsection we will analyse the 
asymptotic behaviour of the solution of Eq.JjHJ as t — > oo. 
Firstly, the equation for the average x(t) is given by: 

fPr r 1 

— = (B + (A~B)x(t))(x2(t)-(x(t)) 2 ), x(0) = J xP (x)dx, (32) 

where x 2 (t) = L x 2 P(x, t)dx. The factor x 2 (t) — (x(t)) 2 is always positive. For 
the four different regions of the (A, B) parameter plane, as illustrated in figure 
1, Eq.JsSJ implies the following. 

(A,B) E I: in this case (B + (A - B)x{t)) > for all t, so x(t) is an in- 
creasing function. Because P(x,t) = for x £ [0, 1], the distribution will 
accumulate at x = 1 and linit_,oo a?(£) = 1. 

(A, B) G 77: Ea. (|32|l now has an attractive fixed point at x = B/(B — A). 
Therefore, lim t _ >00 x(<) = B/(B - A) 

(A,B) 6 ///: similar to /, but now lim t ^ 00 x(t) = 0. 

(A, B) G IV :Ea.l(3"2)) has a repelling fixed point at x = B/(B - A). It follows 
that ]hn.t->oax(t) = if x(0) < B/{B - A) and lim^^ x(t) = 1 if x(0) > 
B/(B-A). 
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In the cases I, III and IV the limiting distributions Poo (a;) = lim^oo P(x, t) 
are point-distributions concentrated on one of the endpoints of the domain [0, 1]. 
This is comparable with the results in the model described in section 3.2 when 
cr^O: eventually all players will play either one or the other of the pure 
strategies. The situation is quite different for case 77, i.e. for Hawk-Dove 
type games. We will show that in this case, Poo(x) depends on the initial 
condition Pq(x), but is in general non-zero on all of [0,1]. This is a dramatic 
difference with the above mentioned model. There, the population is divided 
into a fraction B/(B — A) of the population who play pure strategy I and the 
rest who play strategy II. In our model, where the players have access to a 
continuum of mixed strategies, we do not find that everybody plays the mixed 
strategy x = B/{B — A) (play strategy I with probability B /(B — A)), as might 
be expected. Rather, the final outcome is a population who's members play a 
broad range of strategies, although the average value of the strategies played is 
x = B/{B — A). The details are as follows. 

Every distribution P{x) with xP{x) Ax = x = B j(B — A) is a solution of 
Eq.JSIJ. We will now show that this set of distributions is attractive. 
Let A(t) = (B + (A - B)x(t)). The solution of Eq.lgU then is 



P(x,t) = P (x)exp( X(t')(x-x(t'))dt. (33) 
Jo 

Using L P(x,t)dx = 1 and writing g(t) = jl A(t')dt, it follows that 

exp(/ X(t')x(t')dt)= [ P Q {x)e X9{t) dx. (34) 



Therefore 



Pn(r) P x 9( t ) 

P(x,t)= , 0[X) . (35) 

f^P (x)e^(t)dx 

Differentiating relation (|34J) with respect to t yields 

X(t)x(t)exp( f X(t')x(t')dt) = g\t) f xP {x)e X9{t) dx. 
Jo Jo 

Using g'(t) = X(t), and Eq. (|3*ljl . this reduces to 

L 1 xP (x) e xg ^ dx , . 

x(t) = ^ — ^ . 36 

/„ Po{x) eMt) dx 

Since lim^oo x(t) = B/(B - A), then g(t) = f*(B + (A - B)x(t'))dt' cither 
tends to a finite limit or diverges. Using the change of variable u — x g(t), we 
can write: 

ra(t) , 



1 Jo gW ^o(^)e"du 
3® Io 9it) Po(^)e- du- 



ll 



Therefore, when g(t) — * oo, then 

= Hm £/ 1/£ ^PoM e "d M = Hm g r 1/E Me "du = i 

By a similar argument, when — > — oo then x(t) — > 0. Since lim^oo x(t) — 
Bj (B—A) is neither nor 1, it follows that lim^oo g(t) = go is finite. Therefore, 
as t — > oo the solution Ea. (|35|l tends to 



/ V (i) e^so dx' 

which is clearly not a point-distribution. 

The value of go can be found from Ea. l|36|) . which in the limit t — > oo reads: 



B _ J* xP {x) e xgo dx 
j'l P (x) e x ao dx ' 



B-A 

This equation for go can be solved numerically for given values of A and B and 
a given initial distribution Po(x). In figure 3, two examples, both with A = —3 
and B = 7, are given for different initial distributions, shown in the left column. 
In the right column the final distributions are plotted. 




Figure 3: Initial and final distributions without mutations 



3.3.2 Stationary solutions of the full equation 

Numerical experiments show that all solutions of the full equation l|29|l converge 
to a time-independent solution. The equation for these stationary solutions is 
given by: 

a^P- + (B + (A-B)x)(x-x)P(x) = 0, (37) 
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P'(0) = P'(l) = 0, P(x) > 0, f P(x) dx = 1. (38) 

Jo 

Rather than giving a definition of x in terms of P(x), we take i to be a free 
parameter and impose the conditions (|38|l . Integrating Eq. I|37l) over x from 
to 1 then yields: 

(B + (A — B)x) ( / xP{x) dx-x) = 0. (39) 
Jo 

Assuming that B + (A — B)x ^ 0, the equality 

x = [ xP(x)dx. (40) 
Jo 

then follows automatically. 

Let s(x) = sign(B + (A-B)x) and k(x) = |5±t±rSl|i/3. Then the solution 
of Eq. (|3TJl is given by: 

P(a;) =aAi[—s(x)K(x)(x-x)]+bBi[-s(x)K(x)(x-x)], (41) 

where the Airy functions Ai(z) and Bi(z) are the standard linearly independent 
solutions of y"(z) — zy(z) — 0. These functions are plotted in figure 4. 




Figure 4: Airy functions Ai(z) and Bi(z) 

The position — 770 of the first maximum of Ai(z) is indicated by a vertical 
line segment. The curve for Bi(z) is dashed. Imposing the boundary conditions 
P'(0) = P'(l) = gives 

a Ai [s(x)k(x)x] + b Bi[s(x)k(x)x] = 

aAi'[s(W)n{x)(x-l)] + b Bi'[s{x)k(x){x - 1)] = 0, 

so that a non-trivial solution only exists if x is a solution to the "eigenvalue 
equation" : 

Ai'[s(x)K(x)x]Bi'[s(x)K(x)(x - 1)] - Bi'[s(x)K(x)x]Ai'[s(x)n(x)(x - 1)] = 0. 

(42) 
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Corresponding to a solution x of Eq. H42|) . we find that the solution l|41l) can 
be written as: 



P(x) = (43) 
= c(Bi'[s(x)K(x)x]Ai[s(x)K(x)(x — x)] — Ai'[s(x)K(x)x]Bi[s(x)K(x)(x — x)}), 

in which c is determined by the normalisation condition. We found that, al- 
though the eigenvalue equation l|42[l can have many solutions, there will be 
at most three that correspond to a distribution i|43[l with the property that 
P{x) > for all x £ [0, 1]. For values of A and B in the regions I and II of figure 
1, typified by PD- and HD-games, there is only one solution. The distributions 
calculated for a — 0.001, are shown in figure 5. 



PD 



HD 



Figure 5: Stationary distributions for PD- and HD-games 

For the PD-case the parameters are A = 1, B = 2 and x — 0.901. For 
the HD-case they are A = —2, B = 1 and x = 0.342. For (A,B)eIV, i.e., 
for Coordination Games, there exist three solutions for this value of a. The 
corresponding distributions are plotted in one picture (figure 6). 



Figure 6: Stationary distributions for CG-game 

With A = 2 and B = — 1 we find for the average strategies x — 0.118, 
= 0.323 and x = 0.915. 
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We now study the behaviour of the solution of l|37[) and (|38|l for a — > 0. 
First consider the case that \B + (A — B)~x\ > const, independent of a. Then 
k(x) = | B+ ( A ~ B ^ X |i/3 ^ oo as cr ^ 0. From figure 4 it is clear that if s(x) = 
sign(B + (A — B)x) = 1, then Bi'[s(x)n(x)x] — > oo, Bi'[s(x)«;(x)(x — 1)] remains 
bounded and Ai'[s(x)k{x)x] — > 0, as a — ► 0. This implies that a; can only be a 
solution of Ea.l|42|l if Ai'[s(x)K(x)(x — 1)] — » 0. From figure 4, it then follows 
that — 1) — » —770, where 770 = 1.01879 • • • is the smallest positive solution 

of Ai'{— 770) = 0, so that x — ► 1 and /t(a?) — ► l^l 1 / 3 . More precisely, we have: 

x = 1 - |-^-| 1/3 ?7o as 

This solution is consistent with the assumption s(x) = 1 if and only if A > 0, 
as in the Prisoners Dilemma and the Cooperation Game. The corresponding 
asymptotic expression for P{x) becomes: 

P(x) = cAi[-K(x)(x - x)} = cAi[(— ) 1/3 (1 -x) - 

cr 

Note that for ct — > this distribution becomes sharply peaked at a; = 1, with 
the width of the peak proportional to er 1//3 . 

By a similar reasoning, it is found that for B < (as in the Cooperation 
Game) there exists a solution such that 

x=\-\ 1 / 3 Vo and P(x) = cAi[\-\ 1 / 3 x-r]o} 
B a 

as cr — * 0. 

In the previous section we found that for a = the Hawk-Dove Game and 
the Cooperation Game have solutions with x = B _ A , which we call a central 
solution. This motivates us to look for solutions for which k(x) = \ B +( A - B ) X |V 3 
remains bounded as as a — > 0. We therefore assume that 

*=B^A + B^A a + - aS (44) 

where a 6 i? is as yet unknown. Substituting Ea . (|44|) into Ea. l|42H and taking 
the limit a — ► yields: 

^'[b^" 1731 ^'^" 1731 m ' { ^A aX " W{ ^A aXl ^ = °' (45) 

where a 1 / 3 is understood to mean sign(a) | ct | 1 / 3 . This is the equation from 
which a must be solved. Although Ea. l45fl has many zeroes, only one corre- 
sponds to a positive distribution given by: 

P(x) = c(Bt'[(3a 1/3 }At[a 1/3 {x - 0)] - Ai'[f3a 1/3 }Bi[a 1/3 (x - /?)]), (46) 

with (3 = B B A . For A = — 2 and B = 1, (a Hawk-Dove game), it is plotted in 
figure 7. 
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Figure 7: Asymptotic HD-distribution for a — > 

We note that Eqs. and (|lrj|) are invariant under A — ► —A, £? — » — _B. 

Therefore, as cr — > 0, we find the same central solution for the Coordination 
Game with A = 2 and B = — 1. From numerical simulations we find, however, 
that the central solution is stable in the Hawk-Dove Game, but unstable in the 
Coordination Game. 

Away from the limit a — > 0, we can track the fate of the central solution as a 
grows. Since the full eigenvalue-equation i|42[l is not invariant under A — > —A, 
B — > —B, the situation is different for Hawk-Dove Games as opposed to Co- 
ordination Games. For Hawk-Dove Games, we find that the central solution 
persists for all values of the diffusion coefficient a. However, for Coordination 
Games we find that above a critical value cr c of cr, the unstable central solution 
disappears, together with one of the two stable solutions, leaving only one at- 
tracting, stationary solution. When A < \B\, the solution at x = 1 remains, 
and when A > \B\, the solution near x — survives. This can be summarised 
by saying that for large enough values of cr, the only attracting solution is a 
stationary solution near the risk-dominated solution of the discrete equation. 
The bifurcation process is illustrated in figure 8, where for two values of cr the 
left-hand-side of Eg. (|42|l is plotted as a function of x(a). For cr = 0.0035 this 
function has three zeroes, whereas for a — 0.0039 there is only one. This closely 
resembles the situation in the discrete case as described in section 2. However, 
the critical values for a in the discrete case and in the continuous case are not 
comparable (0.11 vs. 0.0037). 

3.3.3 Summary 

Here we summarise and compare the results of the three types of games, using 
the discrete model, the discrete model with a stochastic term and the continuous 
model with deterministic mutation. 

The Prisoners Dilemma is straightforward. In the discrete case the popula- 
tion will eventually play All Defect. This does not change when the strategy- 
space is made continuous and adding deterministic mutation simply changes the 
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Figure 8: Bifurcation of CG-games 

limiting delta-distribution to a finite peak with width proportional to a 1 / 3 . The 
contrast with the corresponding stochastic equation is that there the width of 
the peak is narrower and proportional to cr 1 / 2 . 

In the discrete version of the Coordination Game, the population will even- 
tually play either x\ = or x\ = 1, where the final outcome depends on the 
intial condition. If the mutation rate is larger than a certain thrcshhold, only 
the stable solution around the risk-dominant solution remains. This result re- 
mains the same in the continuous case. Depending on the initial distribution of 
strategies, the final distribution will be sharply peaked (width of peak propor- 
tional to ct 1 / 3 ) around either x — or x — 1. When the mutation rate becomes 
larger than a critical value, only the distribution around the risk-dominant solu- 
tion remains. We note that this critical value is much smaller in the continuous 
case than in the discrete case. The stochastic equation has only one, attracting, 
stationary solution. When the mutation rate a is small, this distribution is al- 
most completely concentrated around the risk dominant solution, where again 
the width of the distribution is proportional to ct 1 / 2 . 

The Hawk-Dove Game shows the following behaviour. In the discrete case, 
there is an asymptotically stable solution for all values of a > 0, which for 
a = has the value x\ = B/(B — A). The stochastic equation has a unique 
attracting stationary solution, peaked around x = B / (B — A) and with a width 
proportional to a 1 / 2 . Without mutation, a = 0, the continuous equation has 
an asymptotically stable invariant set of solutions, consisting of all distributions 
P(x) with average x = Bj(B — A). Depending on the initial distribution, 
the solution converges to a member of this set. When the mutation term is 
added, a > 0, only one, attracting, solution remains. In the limit a — > 0, 
this solution converges to l|46|). This solution is not concentrated on a single 
point, but has the whole interval [0, 1] as support. Such a solution is sometimes 
referred to as polymorphic. For small ct, numerical experiments show that a 
starting distribution initially converges to a distribution close to the solution it 
would reach if a = 0, but then slowly (on a time-scale of 1/cr) evolves to the 
unique limiting solution. This behaviour resembles that of a singulary perturbed 
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ordinary differential equation, which in the unperturbed case has an attracting 
set of fixed points, and where after adding the perturbation, the attracting 
invariant set survives. In this invariant set we would have, in this analogy, one 
attracting fixed point left. 

4 The Ultimatum Game 

This game has attracted a great deal of interest, mainly as a model to explain 
the occurrence of strong reciprocity in populations of supposedly selfish individ- 
uals. (Binmore 52]) Fehr & Gachter |18|). Here "strong reciprocity" means the 
willingness to share, but also to punish egotistical behaviour in others, even at 
a cost to oneself. See Bowles & Gintis [T5]. 

The game is played by two players. The first player is given a certain amount 
of money and proposes a split of this money with the second player. This second 
player has the choice between accepting the offer, or rejecting it, in which case 
neither of the two players will receive anything. An obvious strategy for the 
second player is to accept every offer, since something is better than nothing. 
Realising this, the first player will maximise his share by offering the lowest 
possible amount to the second player. This (combined) strategy is sometimes 
referred to as "the rational solution" (Page & Nowak J20J), "the subgame-perfect 
equilibrium" (Seymour [21]) or the strategy of "Homo Economicus" (Bowles & 
Gintis (221). 

An evolutionary version of this game, taking into account mutations, was 
studied in 120] and by Nowak et al. [22! , using adaptive dynamics. In adaptive 
dynamics models, the population is assumed to always be monomorphic, i.e., 
everybody plays the same strategy. Every now and then, a mutant is introduced. 
If the strategy of the mutant is more succesful than the resident strategy, it 
will quickly spread in the population, thus becoming the new resident strategy. 
For the Ultimatum Game it was found that in the absence of any restrictions, 
the solution of the adaptive dynamics model indeed converged to the "rational 
solution" . 

The Ultimatum Game was also studied by Seymour |21| . using replicator 
dynamics. He included mutations as a given, exogeneous term and found that 
other solutions can emerge, far from the "subgame-perfect solution", depending 
on the form and intensity of the mutation function. 

We model the Ultimatum Game as follows. The strategy space is S = 
[0, 1] x [0, 1], where a stategy s — (x,y) 6 S means that the player, in the role 
of nr. 1, will offer a fraction x, while in the role of nr. 2 he will reject any offer 
lower than y. 

In one round, the players will play the role of nr. 1 and nr. 2 alternately. 
This leads to a payoff function giving the payoff to strategy s when playing 
strategy s' (the factor 1/2 has been omitted): 

M(s, a') = M(x, y\x', y') = (1 - x)Q(x - y') + x'Q(x' - y), (47) 
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in which the Heaviside function is defined by 

8 M-{ss;<s <48) 

Before writing down the full replicator equation i|14|) for this case we first 
introduce a number of abbreviations: 



H(x,t)= [ P{x,y,t)dy and V(y,t) = [ P(x,y,t)dx, 
Jo Jo 



which are normalised as 



f H(x,t)dx= [ V(y,t)dy = l. 
Jo Jo 

Q(x,t) and R(y,t) are defined by 

Q{x,t) = [ V(y,t)dy and R(y,t) = [ xH(x,t)dx, 

JO Jy 



(49) 



(50) 



(51) 



so that 

Q(0,t) = and Q(l,t) = l and R(l,t) = 0. (52) 
In terms of these functions the local and global averages take the form 

U{x, y, P) = (1 - x)Q(x, t) + R{y, t) (53) 

and 

n(P) = Cl (t) + c 2 (i), (54) 

with 

ci(t)= [ (l-x)Q{x,t)H(x,t)dx CB(i)= / R(y,t)V(y,t)dy. (55) 
Jo Jo 

At last the replicator equation becomes 

dP{X dt V ' t] = [(l- a; )Q( a;> t)+ii( tf ,t)-c 1 (t)-ca(*)]P(i > y,t)+ (56) 
+aAP(x,y,t), 

in which A is the two-dimensional Laplace operator. The boundary condition 
is 

VP • n = on the boundary. (57) 

In what follows we will restrict ourselves to solutions which can be written as 
the product of two normalised functions of (x,t) and of (y,t) respectively. It 
then necessarily follows that 

P(x,y,t) = H(x,t)V(y,t). (58) 



19 



With this restriction we easily show, by integrating Eci. (|56[l over y, that 



dH(x,t) r . . . . , .,„. . d 2 H(x,t) , . 

g t ' = [(1 - x)Q(x, t) - Cl (t)]H(x, t) + a d y (59) 

The boundary conditions are 

dH(x = 0,t) dH(x = l,t) 



dx dx 
Integration over x of Eq. I|56|> leads to 



= for all t. (60) 



d -^ = [R(y,t)-c 2{t )My,t) + a^^, (61) 

with boundary conditions 

dV(y = 0,t) dV(y = l,t) 



dy dy 



for all t. (62) 



We note that for a = these equations are the same as those studied in |2"T] . 
where it is assumed that there are two separate populations of players, one where 
all members always play the role of nr. 1 and the other with nr. 2 players. 

4.1 The equation without mutation 

For cr = the equations become: 
dH(x,t) 



dt 
dV(y,t) 



= [{l-x)Q{x,t)- Cl {t)]H(x,t) 
= [R(y,t)-c 2 (t)]V(y,t). (63) 



dt 

Similar to the Hawk-Dove Game, we identify a set of stationary solutions 



Hq(x) = S(x — x) 

v(y) if y < x 

\i y >x 



V (y) = 



ci — 1 — x , C2 = x (64) 

where v{y) is an arbitrary function with J^v(y)dy — 1 and 8(z) is the Dirac- 
5 distribution. The solution (|64|l is easily checked, by noting that R(y) = 
xQ(x — y) and that Qix) = 1. We have also used zS(z) = 0. The interpretation 
of this solution is clear: player nr. 1 always offers x, so player nr. 2 will always 
receive this amount, as long as his acceptence threshhold is below x. The average 
payoff is therefore II = c\ + C2 = 1 and any distribution of the y- values below x 
is stationary, given this distribution of x. The limit x — > corresponds to the 
subgame-perfect solution. 
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To show that (|64|) represents an attracting set of solutions, we use the same 
reasoning as in section 3.3.1, and find 

exp[/ cx{t')At']= ( H (x) exp[(l - x) [ Q{x,t') dt'] dx = h(t) (65) 
Jo Jo Jo 

H{x, t) = exp [(l - x) J* Q(x, t') dt>] (66) 

exp[ f c 2 (0 dt'} = f V (y) exp[ f R(x, t') dt'] dy = v(t) (67) 
Jo Jo Jo 

V(y 7 t) = y^-eM[ t R(y,t')dt'\. (68) 
v ( t ) Jo 

By differentiating Eq. |R!B|) and ll6"7j) . respectively, we obtain: 



1 



l 



Cl{t) = W)J ^- x ^ x ^ H ^ e ^ l ~ x n Q(M')ch']dx 



(i 



oa(*) = -L/ R(y,t)V (y) exp[ f R(y,t')dt']dy. (70) 
u l r J Jo Jo 

Assuming that V(y, t) converges to a stationary distribution (as all numerical 
results show), then (1 — x) J Q Q(x,t')dt' converges to a function with a finite 
number of isolated local maxima. One of these, say x = x, is the absolute 
maximum, and it follows from Ea. H66|) that H(x,t) converges to 8{x — x). From 
Eq.JHU it follows that R(y,t) converges to R(y) — xQ(x — y) and V(y,t) con- 
verges to cVo{y)Q(x — y), with c a normalization constant. From Ea. ()55j) it 
follows that Ci(t) converges to 1 — 3? and Ci{b) to x. 

The above considerations show that if the solution of Eq. l|63|l converges to a 
stationary solution, it must be a member of the invariant set l)64[l. However, the 
value of x cannot be predicted from the above formula's. Numerical solution 
of Ea. 1)63(1 shows that for random initial distributions of H(x, 0) and V(y, 0) on 
the whole interval [0,1], the functions H(x,t) and V(y,t) indeed approach the 
form of Ea. l|64|l for t — > oo. The average strategy x, based on 100 simulations, 
takes values between 0.12 and 0.30, with a mean value equal to 0.22 and a stan- 
dard deviation of 0.04. We note that a uniform distribution of both H (x, 0) and 
V(y, 0) also leads to a value of x = 0.22. 

4.2 The equation with mutation 

We have found numerically that when a > 0, all solutions of the full equations 
l|59|) and l|61|l tend to a unique solution of the stationary equations: 

a^^ + [(l-x)Q(x)- Cl ]H(x)=0 (71) 
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and 

v^-^ + lR(y)-c 2 }V(y)=0. (72) 
The boundary values are those of Eas. (|60|) and (|62[) . 

Unfortunately, we have not been able to find closed form expressions for the 
solutions of these equations. There are two ways to approximate the stationary 
solution, which lead to the same result. First, we numerically solved the full 
equations (|59(l and (jfiipi . by discretising space, solving the resulting coupled set 
of ordinary differential equations and considering the solution as t — > oo. In the 
second method, we define the following seven functions 

F x {x) = H(x), F 2 (x) = ^-, F 3 (x) = V(x), Fi (x) = ^-, 

F 5 (x) = Q(x), F 6 (x)=R{x), F 7 (x)= [ H{x')dx'. (73) 



In terms of these functions and with k = l/a .the stationary equations can now 
be written as 



dF 1 (x) 

dx 
dF 2 {x) 

dx 
dF 3 {x) 

dx 
dF 4 (x) 

dx 
dF 5 (x) 

dx 
dF 6 {x) 

dx 
dF 7 (x) 

dx 



F 2 (x) 

-k[(l - x)F 5 {x) -c x ]F x (x) 
F A (x) 

-k[F 6 {x) - c 2 ]F 3 (x) (74) 

F S (x) 

-xF x (x) 

-F x (x) 



These equation can be solved numerically by starting the integration from the 
following values at x — 1 

F X {1) = a, F 2 (1) = 0, F 3 (l) = b, F 4 (1) = 0, 

F 5 (l) = 1, F 6 (1)=0, F 7 (1)=0, (75) 

and using a standard routine to arrive at the values of these functions in x = 0. 
The numbers (a,b,c x ,c 2 ) are as yet unknown. They should be chosen in such a 
way that the boundary conditions at x = be satisfied, i.e., 

(F 2 (0),F 4 (0),F 5 (0),F 7 (0)) = (0,0,0,1). (76) 

This matching of four numbers by varying four other numbers should be possible 
in many ways. It turns out, however, that the requirement of positivity of F x (x) 
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and Fs(x) in the whole interval [0, 1] makes the solution unique. A root finding 
routine of Mathematica does the job. For a = 0.001 the stationary solutions 
H(x) and V(y) are shown in figure 9. 



H 



V 




1 




1 



Figure 9: Stationary solution of Eas. ljTT|l and JTSJ for a = 0.001 

We note that H(x) has a Gauss-like distribution around a mean value x = 
0.3172, while V(y) is approximated by the right half of a Gaussian, with its 
maximum at y = 0. For smaller values of a, the value of x(a) and the width of 
the peak of the if (^-distribution decrease, but the shape of the distributions 
is otherwise unchanged. 

For values of a down to 10 -9 we have calculated the cr-dependence of x. In 
figure 10 



a log-log plot of this dependence is shown. A good fit of the data points is 
given by x(a) « 1.64 x a 23 . 

The numerical solution of the full time dependent equations reveals a dynam- 
ical pattern similar to the Hawk-Dove Game of section 2. Initially a distribution 
approaches the attracting set Ij64(l , after which it slowly converges to the unique 
stationary solution, on a time-scale of l/cr . 

5 Conclusions 

In this paper we have generalised the replicator dynamics of games with de- 
terministic mutations, as described in [3], to the situation where players have 




Figure 10: Log- log plot of x as function of a 
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access to a continuous strategy space. The resulting equation (|T4l has a well 
defined solution, which, however, is not easy to analyse in general. 

Our first example, the continuous version of 2 x 2 symmetric games, already 
illustrates a number of interesting and perhaps unexpected features of this equa- 
tion. Although there is no a priori reason to believe that the continuous and 
the discrete strategy version of the same 2x2 game have anything to do with 
each other, the similarities between some results warrant our surprise at the 
differences in others. The continuous Prisoners Dilemma and the Coordination 
Game behave similar to their discrete case counterparts: the final state is a 
monomorphic distribution, where every player in the population plays one of 
the two pure strategies. The extension to a continuous strategy space and the 
inclusion of mutation (which has the form of a diffusion term) only leads to 
the existence of small variations around the single peak of the final distribution. 
Also in the continuous Coordination Game we encounter, as in the discrete case, 
a threshold value for the mutation term separating a regime with two attracting 
solutions from one where only a single attractor exists. 

The difference occurs in the Hawk-Dove game. In the discrete version there 
is a stable equilibrium with Hawks and Doves coexisting. In the continuous ver- 
sion this does not translate into a monomorphic distribution around the mixed 
strategy corresponding to this equilibrium. Rather, in the unique limiting dis- 
tribution the whole range of mixed strategies from pure Hawk to pure Dove 
is represented. The attraction to this stationary solution occurs on two time 
scales. On a fast time scale, the solution is attracted to the set of distributions 
with average corresponding to the equilibrium mixed strategy of the discrete 
case. Then, on a slow time scale proportional to the inverse of the mutation 
rate, the solution converges to the unique attractor. This two timescales phe- 
nomenon was observed in numerical simulations, and is currently awaiting a 
more thorough analysis. Also, there are many interesting games with three or 
more strategies (for instance Rock, Scissors, Paper) of which the continuous 
version can hold more surprises. 

The results of the second example, the Ultimatum Game, are of great inter- 
est to the debate around strong reciprocity and how it could have evolved. Our 
model shows that replicator dynamics and a small mutation term can lead to 
a final outcome far from the subgame-perfect solution. Take, for instance, a 
mutation rate of 10 -3 . So in every time-interval, all players vary their strategies 
according to the cold rules of self-interest, after which a small fraction of 0.1% 
of the population, change their strategy just a little bit. Then we find that an 
initial population consisting almost entirely of cynical misers (accept everything 
and offer nothing) , eventually turns into a world where the average offer is more 
than 30%! 

This surprising result can be explained in the following way. Consider a 
situation where all proposers offer only a small share to their opponent and 
these opponents all have an acceptance threshhold lower than this offer. Now, 
due to mutation, some acceptors will demand a share that is slightly larger than 
what is being offered. Normally, this would be a suicidal strategy. However, 
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also due to mutation, there will be amongst the proposers a small set who are 
willing to offer slightly more than their colleagues. On the one hand, these 
fairer-minded proposers earn slightly less from the bulk of the acceptors, but on 
the other hand they are the only ones to profit from the small group of high- 
minded mutants on the acceptor-side. The net result can be that the second 
effect dominates and that there will be a tendency towards higher offers. 

For sufficiently small mutation rates the dynamics of the Ultimatum Game 
show the same structure with two timescales as the continuous Hawk-Dove 
game. An initial distribution is quickly attracted to a distribution where the 
offers are sharply peaked, and then slowly converges to the unique stationary 
solution. 

In this case too, a more rigorous mathematical analysis is required for a 
better understanding of the model. In particular it would be nice to be able 
to calculate the value of the exponent in the formula relating the mutation 
rate and the average value of the offers, which in this paper we derived from 
numerical simulations. For this purpose singular perturbation theory seems to 
be an appropriate tool. Furthermore, we have only considered mutation rates 
that are the same for the proposers as for the acceptors. Differentiating between 
these may also lead to a fuller understanding. 
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